A Revisit of Block Power Methods for Finite State Markov Chain Applications

According to the fundamental theory of Markov chains, under a simple connectedness condition, iteration of a Markov transition matrix P , on any random initial state probability vector, will converge to a unique stationary distribution, whose probability vector corresponds to the left eigenvector associated with the dominant eigenvalue λ1 = 1. The corresponding convergence speed is governed by the magnitude of the second dominant eigenvalue λ2 of P . Due to the simplicity to implement, this approach by using power iterations to approximate the dominant eigenpair, known as the power method, is often applied to estimate the stationary distributions of finite state Markov chains in many applications. In this paper, we revisit the generalized block power methods for approximating the eigenvector associated with λ1 = 1 of a Markov chain transition matrix. Our analysis of the block power method shows that when s linearly independent probability vectors are used as the initial block, the convergence of the block power method to the stationary distribution depends on the magnitude of the (s+1)th dominant eigenvalue λs+1 of P instead of that of λ2 in the power method. Therefore, the block power method with block size s is particularly effective for transition matrices where |λs+1| is well separated from λ1 = 1 but |λ2| is not. This approach is particularly useful when visiting the elements of a large transition matrix is the main computational bottleneck over matrix–vector multiplications, where the block power method can effectively reduce the total number of times to pass over the matrix. To further reduce the overall computational cost, we combine the block power method with a sliding window scheme, taking advantage of the subsequent vectors of the latest s iterations to assemble the block matrix. The sliding window scheme correlates vectors in the sliding window to quickly remove the influences from the eigenvalues whose magnitudes are smaller than |λs| to reduce the overall number of matrix–vector multiplications to reach convergence. Finally, we compare the effectiveness of these methods in a Markov chain model representing a stochastic luminal calcium release site.